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Abstract. The quality of education is 
increasingly being measured less by the 
knowledge gained during schooling and 
more by the level of competence pos- 
sessed by students at doing a particular job 
after completing their education. Target 
and process-planned curricula are being 
replaced more and more by competence- 
oriented curricula, especially in science and 
technology education, where competences, 
generally defined as the capability or the 
ability of an individual doing a job properly, 
are placed at the forefront. 

If competence is the desired criterion 

for educational quality, it can be easily 
established that suitable instruments and 
methods of measurement are needed 

for this kind of quality evaluation, which, 
however, are not yet available. Suitable in- 
strumentation was developed in this study. 
Its use was demonstrated in the example of 
elementary education in Slovenia. 
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Introduction 


More and more countries are focusing on measuring and monitoring the 
quality of education. This holds true for Slovenia as well, where the Ministry 
of Education started reviewing and upgrading its educational policy between 
2012 and 2013. It is clear that small corrections alone will not suffice, and 
that the entire education paradigm must be changed; it is also important 
to recognise that this cannot be done in one, simple step, but rather in the 
form of a long-term process. 

One of the certain consequences of a non-competitive education system 
is high unemployment among the young, and the lowering of the general 
standard of living (Dumont, Istance, Benavides, 2010). These parameters are 
included in the regular reports made by international organisations such as 
UNESCO and OECD. Also, the Education for All (EFA) report “Regional over- 
view: Central and Eastern Europe and Central Asia” emphasises the urgency 
in raising the competences needed for jobs in the 21* Century. Education 
and the method of schooling and studying do not provide the young with 
a competitive advantage, i.e. employability. Therefore, in todays society the 
young must be equipped with fundamental competences that are general 
in character (e.g. learning how to learn) and, thus, transferable between 
different areas. Skills and technical-vocational knowledge must be added 
to this (UNESCO, 2007). The results of the Progress in International Reading 
Literacy Study (PIRLS) suggest that the model of quality teaching is composed 
of three fundamental dimensions, with each of them being composed of 6 
elements: 


e Intellectual dimension (Elements: deep knowledge, deep un- 
derstanding, problematic knowledge, higher-order thinking, 
metalanguage, substantive communication) 

e Learning environment (Elements: explicit quality criteria, commit- 


Journal of Baltic Science Education, Vol. 13, No. 5, 2014 


ISSN 1648-3898 DEVELOPMENT AND EVALUATION OF A COMPETENGE-BASED TEACHING PROGESS FOR SCIENCE 
AND TECHNOLOGY EDUCATION 
(P. 740-755) 


ment, high expectations, mutual support, students’ self-control, student-teacher joint decisions), and 
e =Making learning meaningful (Elements: prior knowledge, cultural sophistication, knowledge integra- 
tion, inclusion, narration). 


In analysing and comparing different international studies, it is important to consider the fact that the differ- 
ences in student performance across OECD member countries are explained by the following factors (UNESCO, 
2008): 


e 55% by student characteristics, 
e 34% by environmental factors influence, 
e 11% by differences between countries. 


To analyse the current state of the educational system in Slovenia it makes sense to take into account, in ad- 
dition to UNESCO's reports, some of the key international studies such as the Programme for International Student 
Assessment (PISA), International Computer and Information Literacy Study (ICILS), OECD’s Teaching and Learning 
International Survey (TALIS), and the international survey Health Behaviour In School-Aged Children, conducted 
under the auspices of the OECD (HBSC). 


Competency for 21 Century 


The term “competency” first appeared in an article written by R.W. White (1959) in 1959 as a concept for 
performance motivation. Later, in 1970, Craig C. Lundberg (1970) defined the concept in “Planning the Executive 
Development Program”. As it is quite a popular concept, there are many definitions about what competency could 
be. Generally, a competency is defined as the capability or the ability of an individual to do a job properly. It is a 
set of related but different sets of behaviour organised around an underlying construct, which we call the “intent”. 
Competency includes the following elements: 


COMPETENCY 


KNOWLEDGE SKILLS ATTRIBUTE 
Relates to information | Relates to the ablity to Relates to qualitative Outstanding 
do aspects Performance of task or 
Cognitive domain Physical Domain Personal characteristic activities 
or traits 





Figure 1; Elements of competency. 


In accordance with this concept competence could also be called procedural and strategic skills. Thus, the 
authors Romainville, Coolahan, Weinert and Day defined competencies as the general capacity to act and the 
ability to perform tasks and activities based on knowledge, experience and values (Key Competencies, 2002). 
From numerous analyses and definitions of competencies it can be summarized that competencies are the skills, 
abilities, knowledge and experience of an individual to perform certain tasks and are a result of their personal 
development, mainly as a result of learning. 

There is also great diversity regarding naming different competencies. Thus, the most frequent concepts 
are: key competencies, generic and subject-specific competencies. Generic competencies are those which an 
individual develops through typical approaches, procedures and strategies of educational work rather than with 
specific learning of a certain matter. The basis for the creation of a set of generic competencies was the Report of 
Mayer's Committee (1991). Generic competencies in the field of science, including technical science, are defined 
as competencies that the individual develops through the method of work rather than with the specific learning 
of a certain matter (Key Competencies Supplementary Statement for Engineering Technology, 2001). 
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Competency Development 


Dreyfus and Dreyfus (1980) introduced a nomenclature for the levels of competence in competency develop- 
ment. The process of competency development is a lifelong series of doing and reflecting. As competencies apply 
to careers as well as jobs, lifelong competency development is linked with personal development as a management 
concept. The four general areas of competency are: 

1. Meaning Competency: The person assessed must be able to identify with the purpose of the organisa- 
tion or community and act from the preferred future in accordance with the values of the organisation 
or community. 

2. Relation Competency: The ability to create and nurture connections with the stakeholders of the primary 
tasks must be shown. 

3. Learning Competency: The person assessed must be able to create and look for situations that make it 
possible to experiment with the set of solutions that make it possible to complete the primary tasks 
and reflect on the experience. 

4, Change Competency: The person assessed must be able to act in new ways when it will promote the 
purpose of the organisation or community and make the preferred future come to life. 

The first two areas are particularly important in real working situations; therefore the focus will be on two 
competencies which are very important in schools, the learning and the change competency, with a special 
emphasis on skills from Figure 1. This entails the need for the following skills in today’s schools, shown in Table 1 
(Anderson, 2010). 


Table 1. | Competencies for the 21* century. 


Competencies for the 21st Century 


Analytical skills Interpersonal skills The ability of realizing mrernanen pomye a anging/ 

processing learning 
Critical thinking oman tow rns Information literacy : eae 
Messaging self-regulation innovation 

Problem solving Collaborating Productivity, efficiency Media literacy Adaptability/ learning to learn 
Decision-making Leading and responsibility Digital citizenship Flexibility 

Research and develop- ICT procedure and 
ment concepts 


Every 21% Century skills implementation requires the development of core subject knowledge and understand- 
ing among all students. Those who can think critically and communicate effectively must build on a base of core 
academic subject knowledge. Within the context of core knowledge instruction, students must also learn the es- 
sential skills for success in today’s world, such as critical thinking, problem solving, communication and collaboration. 
In the research presented, the development and evaluation of life and learning and innovation skills will be the 
subject of research attention. 
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21st Century Student Outcomes and Support Systems 





Learning an Innovation Skills - 4Cs 
Critical thinking, Communication, 
Colloboration, Creativity 


Life and Information, 
Career Skills Core Subjects - 3Rs Media, and 
and 21st Century Themes Technology 

Skills 





Standard and Assesment 
Curriculum and Instruction 


Professional Development 


Learning enviroments 





Figure 2: Students’ outcome and support system. 


Generic Competency 


Generic competency is also known as object-independent and transversal or transferable competency, respec- 
tively. The creation of generic competencies depends on the specific area. When designing generic competencies 
in the field of science education, conceptual (content) and technical (design) aspects were considered; those were 
provided by the working group in order to identify basic concepts common to all natural science disciplines. 

Skills as a part of generic competencies need to be developed throughout life, not just because of personal 
fulfilment and in order to actively engage with the society in which we live, but also so that we can be successful 
in a working world that is constantly changing. These are the most important factors in innovation, productivity 
and competitiveness (Figel, 2007). 

In an educational process based on a competence-based curriculum, life and learning and innovation skills that 
influence the development of an individual generic competence are at the forefront. Students should therefore be 
able to use tools for the production, presentation and understanding of complex information, and have the ability 
to access, search and use internet-based services. They should also be able to use information and communications 
technology (ICT) to support critical thinking, creativity and innovation. 

The generic competencies were the starting point for the observation and evaluation of students’ skills in 
this study. 


Teaching Methods to Support Competence-Based Teaching Process 


Three types of learning based on research will probably prevail in competence-based teaching in the 21* 
Century: project-based, problem-based and research-based learning (Barron, et. all., 1998, Aber ek, 2012). These will 
provide students with more comprehensive learning in order to be able to use competences obtained in school 
(knowledge, skills etc.) for problem-solving in real life. Research-based approaches are important for nurturing 
communication, collaboration, creativity and critical thinking. It must be noted, however, that research-based 
learning is greatly dependent on well-structured assessment as regards defining learning tasks and the evaluation 
of learned content. This will be discussed in detail later. The success of research approaches is, of course, very much 
dependent on the knowledge and skills of those performing them. It is therefore not amiss out of place to note 
that all such reforms must include teacher training before they commence work in schools. 
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Problem of Research 


In 1996, for the first time in the new state, Slovenia adopted an Elementary School Act and, on the basis of 
the concepts contained in The White Paper on Education in the Republic of Slovenia (1996), introduced compre- 
hensive regulation at all levels of the educational system. In addition to the concept of knowledge, the concept of 
competence began to establish itself in the school environment, and was for the first time seen as knowledge and 
skill. The concept of competence has drawn attention to the complexity of knowledge, its entanglement with the 
disposition of the individual and the need to teach the use of skills (The White Paper, 2011). 

Teaching on the basis of competencies should take place at all stages of the teaching process, from planning 
and implementation to evaluation, since this is the only way to achieve the objective of competent education - a 
comprehensive competent person. Competence diagnostics, through which teachers and students can find out 
to what degree an individual's competence is developed, plays a vital part, which lead to the development of a 
competitive model as a diagnostic instrument (Upheus, 2010). Evaluation and self-evaluation of performances 
provide a basis for the individual student’s results (portfolio), which can also serve as an incentive for further work. 
“Instruments for self-evaluation, which are offered in some school textbooks, represent an important step towards 
the diagnosis of competences” (Hieber, Lenz, Stengelin, 2011: 3). As noted by R. Wesselink et. al., “the design of a 
competency-based curriculum, the design of the learning process and the design process of evaluation can be per- 
formed only when the concept of competencies is defined as clearly as possible. Another great trap is assessment, 
which is viewed as the Achilles heel of competency-based learning” (Wesselink, 2007: 39). Yet to be able to measure 
specific competencies they should be standardised so that they can be recognised and become measurable. 

Knowledge and attitudes (according to Bloom's Cognitive and Affective Taxonomies (Bloom, 1956, Anderson, 
Krathwohl, 2001) can be measured with standardised tests. To assess knowledge in the field of technology, tests 
for the National Assessment of Knowledge are used in Slovenia and are the same for the entire country. Attitudes 
can also be assessed by using standardised procedures, such as the Thurstone, Likert (Judd, Smith, Kidder, 1991) 
or Bogardus scale (Miller, 1991). A problem in assessing skills arises when there are no standardised tests. With 
the introduction of a nine-year elementary school, the developers of the curriculum emphasised that the total 
grade should also reflect the grade in experimental work, i.e. assessment of skills, but in practice this proved to 
be extremely difficult, if not almost impossible. The most commonly stated reason for this was that the number of 
students to be assessed was too high. The problem also lay in the measurement methods and instruments, which 
should have been verified and uniform for a specific area and should also have enabled quick and objective assess- 
ment of students’ skills. This study included the use of methods and instruments that enable suitable development 
and evaluation of students’ skills. 


Research Methodology 
General Research Background 


Special methods and instruments were developed for the evaluation of student competencies in this research. 
The methods and instruments developed can be further improved on the basis of the results obtained from the 
study. The research was performed in two consecutive school years, 2011/2012 and 2012/2013. 

In the 2011/2012 school year the first study was carried out and was used to verify the basic measurement 
characteristics of the test and the selected instrument, the so-called recording form (see Table 2). The recording 
form was used by an external expert to grade (circle) the level of mastering a specific skill at both the lower and 
higher taxonomic levels. Students were divided into two groups, an experimental and a control group. Work in 
the experimental group was project-based and included problem-based and research-based lessons, an experi- 
ment and a technical analysis. Students drilled and practised specific skills before being assessed. In the control 
group the lessons were taught conventionally, with a teacher providing explanations and teaching in front of the 
classroom, and working with textbooks. There was no drilling of a specific skill. The study was performed in order 
to verify the validity, reliability, objectivity and sensitivity of the recording form. On the basis of this preliminary 
pilot study and a comprehensive analysis of the instruments used and the learning process it was established that 
the study could continue. 

The second study was carried out in the 2012/2013 school year. In the experimental group the method of 
teaching was used as the experimental factor and gender and initial assessment in natural science (final grade in 
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Science and Technology in the 5" grade) as the non-experimental factor. In the control group gender and final 
grade in Science and Technology in the 5" grade were used as the non-experimental factor. No experimental factor 
was used since the lessons were taught in a traditional way. 

At this stage of the study the emphasis was put on developing and assessing students’ skills. The procedure 
was identical to that in the preceding stage. Among others, the main purpose of the study was also to establish 
whether project-based learning provides better conditions for developing students’ skills than traditional instruc- 
tion. The variables in the pedagogical experiment were: 

e Independent variables: Gender, grade, experimental group, control group, taxonomic levels and teach- 
ing methods; 

e Dependent variables: Initial state of skills at a lower taxonomic levels, initial state of skills at a higher 
taxonomic level, final state of skills at lower taxonomic levels, final balance of skills at a higher taxo- 
nomic level. 

The data were computer processed using the SPSS 20,0 program for statistical analysis, at the descriptive and 
inferential statistics levels. The following procedures were used: 

e frequency distributions (f, f%) of descriptive variables (gender and final grade); 

e  y’-test of the hypothesis of independence to test dependent associations between variables (according 
to gender and the final grade in Science and Technology); 

e _T-test for verifying hypotheses in smaller samples (progress dependent on gender and final grade). 


Research Sample 


During the 2011/2012 school year the study included 38 students from the 6" grade, aged between 10 and 11. 
They were randomly divided into two groups, the control group - CG (20 students) and the experimental group - EG 
(18 students). The research sample included approximately the same number of boys and girls. 

The second study was carried out in the 2012/2013 school year. The study included 35 students from the 6" 
grade, aged between 10 and 11. They were randomly selected (regardless of their grade and gender) and divided 
into two groups, the experimental (17 students) and the control group (18 students). There were a third as many 
boys in the research sample as girls. 


Instruments and Procedures 
Content characteristics of the measurement instrument 


A recording form (Table 2) was used for direct observation of students’ skills in carrying out a specific task. 
The name of the generic competence was written in the form’s title field. That was followed by a definition of the 
observed skill, how the skill was observed and what had to be taken into account. Mastery of specific skills was 
monitored and evaluated according to the criteria for each taxonomic level: knowledge, comprehension, applica- 
tion, analysis, synthesis and evaluation. These could be adapted to the individual subject and thematic area. The 
observer had to circle the degree of control, where 1 meant unsatisfactory, 2 satisfactory, 3 good, 4 fairly good 
and 5 very good mastery of the skill. 
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Table 2: Example of a recording form. 


Competence: ABILITY TO INTERPRET INFORMATION 


SKILLS: 
e Presentation of information in a clear manner, correct use of professional terms; 
e Preparation and presentation of a project (work plan, seminar or poster). 


HOW? Observe the skills of the particular student and circle the relevant level of skill mastery. 
Pay attention to whether the information is presented clearly and accurately, and whether the student had prepared the 
project independently. 


CRITERION MASTERY OF SKILL 


In their presentation the student reviewed already known facts and 
concepts. The project was not elaborated on; other students could not 
discern the point. 


The student presented information in a clear way and reported inde- 
pendently using professional terms yet phrased in the student's own 
words. The project was elaborated on and summarised the essential 
points. 


HIGHER 
LEVEL 


Key elements were connected in the presentation. The student inde- 
pendently formed clear conclusions that were correct and reliable. The 
student's own ideas were dominant. The student evaluated their project 
and suggested improvements. 





Procedure 


In this research were used two methods, triangulation method and qualitative educational research method. 
The purpose of triangulation in qualitative research is to increase the credibility and validity of the results (Bog- 
dan, Biklen, 2006). In this case triangulation was used to observe the process and verify the students’ attainment 
of skills. This ensured that the research situation could be observed from three perspectives: the teacher - the 
performer, student and observer. The observer was an expert in the field of technical education. The method of 
direct observation was used. 
With the direct observation of skills - social, communication and working skills, we have determined the initial 
state of specific skills in the experimental and the control group: 
e skills at lower taxonomic levels, and 
e skills at higher taxonomic levels. 


The following table (Table 3) shows students’ skills that were developed and assessed at lower and higher 
taxonomic levels. 
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Table 3. 


SKILLS 


COMMUNICA- 
TION 
SKILLS 


SOCIAL 
SKILLS 


WORKING 
SKILLS 


COMPETENCE 


Ability to collect 
information 


Ability to analyse 
and organise 
information 


Ability to interpret 


Ability to synthe- 
size conclusions 


Ability to solve 
problems 


Verbal and written 
communication 


Ability for inde- 
pendent and team 
work 


Organisation and 
planning of work 


Interpersonal 
interaction 


Transferring 
theory into 
practice 


Use of math- 
ematical ideas 
and techniques 


Adapting to new 
situations 


Concern for 
quality 


Safety at work 


LOWER LEVEL 
Student: 


Information is collected using a previously 
presented method; only one source is 
used. 


The information is verified for accuracy 
and organised in previously presented 
structures. 


Information is provided in known contexts. 
Information is presented in a clear manner 
and independently. 


The process of connecting data and 
forming conclusions is reviewed. Results 
are presented using only previously taught 
procedures. 


Guidelines for achieving pre-set goals are 
considered and the results are verified 
using set criteria. 


Ability to describe the work procedure and 
to orally present their work/project. 


Understanding of the task is demonstrated 
and a suitable group work method is 
selected. The student takes on a specific 
group role and understands that role. 


The sequence of work procedures is 
reviewed and a proposed plan of the work 
is described. 


Communication within the group is 
established and characteristics of that 
communication (frequent, rare) are 
explained; the student describes whether 
their ideas and opinions and suggestions 
by other group members were taken into 
consideration. 


Workplaces, tools, machinery and equip- 
ment needed to implement the task are 
prepared. 


Guidelines for selecting ideas and tech- 
niques for specific tasks are observed. 
Mathematical knowledge is used in known 
circumstances. 


Already known processing procedures are 
used with new material. 


Presented equipment is used for 
increased product quality. 


Safety equipment is used when working 
with tools and machinery; the student 
understands the importance of quick and 
correct action in the event of danger. 
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Students’ skills at lower and higher taxonomic levels. 


HIGHER LEVEL 
Student: 


Electronic and written sources are used to search for informa- 
tion. 


The obtained information is verified and connected into a 
meaningful structure. The quality and validity of information are 
assessed. 


Key elements are included in the presentation. Clear conclu- 
sions, which are correct and reliable, are formed independently. 
The student's own ideas are dominant. The student evaluates 
their presentation and suggests improvements. 


Different types of graphs are used to present information; simi- 
larities and differences in each type are pointed out. Tables and 
graphs are modified/rearranged; their use is assessed. 


Student’s own goals are set and the methods of achieving these 
goals are mastered. Own criteria for evaluation procedures are 
set and used, and own criteria for result evaluation are used. 


Work/project’s key elements - words are emphasised, and infor- 
mation is transmitted to other students in a clear, professional 
and logical manner. 


The purpose and goals of the task are set, and how they will be 
realised is described. The roles, procedures and time-frames 

in the group are assessed on the basis of various aspects. 
Negotiating with other group members to set the goals that need 
to be verified and defined. 


Independent performance of procedures according to instruc- 
tions, planning and organising work, finding key elements for 
good planning and work organisation. The student checks 
whether the work performed complies with the work planned, 
and assesses the individuals’ work. 


The similarities and differences of the proposals made by the 
group members’ are presented and the reasons for the deci- 
sions made. The importance of collaboration and communication 
between members of the group and other groups is evaluated. 


The importance of the correct sequence of workplaces and the 
right use of tools, machinery and equipment is explained. A 
proposal is made for solutions regarding rationalisation of work 
places and procedures. 


The assumptions needed for the selection and use of math- 
ematical ideas and techniques are recognised. Mathematical 
knowledge is adapted and used correctly in new circumstances. 
Problems are anticipated and measures taken to eliminate them. 


Processing procedures and tools for new material are selected, 
tested, and the most suitable procedures and tools are used. 


Student’s own equipment is made and used for greater product 
quality, and suggestions are made for improvement. 


Devices, machines and tools are used safely; quick and correct 
action is taken in the event of danger or risk. The efficiency of 
safety equipment is verified. 
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In the experimental group project-based learning (PBL) was used. The training and demonstration of certain 
skills were conducted. Students trained in all three specific skills. In the control group, a frontal teaching method 
was used, e.g. conversation, demonstration and work with texts. There was no training in specific types of skills. 

The instrument and measurement methodologies developed enabled the optimisation of the learning 
process; different methods of work were assessed depending on the given objectives with a focus on students’ 
working, communication and social skills. 


Results of the Research 


a) Gender and final grade in Science and Technology (hereinafter S&T) in the 5" grade’. 
At the beginning, the homogeneity of the control and the experimental group had to be verified. The differ- 
ence between both groups in gender and in final grade in S&T in the 5" grade was examined, as shown in Tables 


4and5. 


Table 4. Student gender in the experimental and control groups. 


Group 
Total 
Experimental (EG) Control (CG) 

Boys No. 11 12 23 

% 64.7 66.7 65.7 
Student gender 

Girls No. 6 6 12 
% 35.3 33:3 34.3 

Total No. 17 18 30 
% 100.0 100.0 100.0 


p=0.903 ?=0.015 


It was established that there was no statistically significant difference between the experimental (EG) and the 
control group (CG) as regards gender (P = 0.903, y? = 0.015). The same was true for the final grade in S&T since the 
experimental group's average grade was 4.11 and the control group's 4.00 (P = 0.891, y* = 0.23), as shown in more 
detail in Table 5. Consequently, the composition of both groups was statistically appropriate. 


b) Initial state of the experimental and the control group 


Whether the didactic approach (work forms and methods) affected the development of skills as part of the 
students’ generic competences was also considered. Therefore, the mastery of skills before the start of the experi- 
ment had to be examined. The following analysis shows the initial level of students’ mastery of a specific skill for 
the lower (Table 6) and the higher taxonomic levels (Table 7). Students’ skills were divided into three groups: social, 
communication and working skills, as shown in Table 3. 


1 S&T - Science and technology is a subject in the 5“ grade of elementary school that includes topics from natural sciences and technology. 
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Table 5. Final grade in S&T in the 5" grade. 


Group Total 
Student gender oe 
EG CG 
C No. 6 a 10 
% 45.5 41.7 43.5 
, , No. 5 4) 10 
Final grade in S&T B 0f, A5 5 AV] 43.5 
Boys 
A No. 1 2 3 
% 9.1 16.7 13.0 
Total No. 11 12 23 
% 100.0 100.0 100.0 
C No. 0 1 1 
% 0.0 16.7 8.3 
,; , No. 0 1 1 
Final grade in S&T B 0f, 0.0 16.7 33 
Girls 
A No. 6 4 10 
% 100.0 66.7 83.3 
Total No. 6 6 12 
% 100.0 100.0 100.0 
C No. 5 6 11 
% 29.4 30.0 31.4 
; No. a 6 11 
Final grade in S&T B 9p, 59 A 333 344 
Total 
A No. 7 6 13 
% 41.2 RRR) ore 
Total No. 17 18 35 
% 100.0 100.0 100.0 
p=0.891 2=0.23 
Table 6. Initial state of skill mastery - lower level. 
GROUP Arithmetic Standard Test for difference of Test for homogeneity of 
mean deviation means variants 
SKILLS x S Boys Girls Boys Girls 
SOCIAL EG 3.470 0.750 
CG 3.324 0.735 
COMMUNICATION EG 2.269 0.643 0.470 0.473 0.353 0.657 
CG 2.115 0.552 
WORKING EG 3.447 0.605 
CG 3.300 0.593 
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Table 7. Initial state of skill mastery - higher level. 
GROUP Arithmetic Standard Test for difference of Test for homogeneity of 
mean deviation means variants 
SKILLS x S Boys Girls Boys Girls 
SOCIAL EG 2.921 0.740 
CG 2.675 0.788 
COMMUNICATION EG 1.509 0.535 0.499 0.284 0.318 0.577 
CG 1.370 0.490 
WORKING EG 2.388 0.719 
CG 2.133 0.665 


The test for difference of arithmetic means showed the students’ mastery of communication skills (written 
and oral communication) at the lower and at the higher level to be the poorest; however, the analysis showed that 
there was no significant difference between skill mastery at the lower and higher level between the experimental 
and control group (Boys = 0.470, Girls = 0.473 — lower level, Boys = 0.499 Girls = 0.284 - higher level). 

Figure 3 shows the level of mastery of a specific skill at the lower and the higher level. The level of students’ 
skill mastery is represented by an ordinate encompassing values from 1 to 5, where 1 means unsatisfactory, 1.5 - 2 
satisfactory, 2.5 - 3 good, 3.5 - 4 quite good and 4.5 - 5 very good. The most problems arose in the field of written 
and oral communication due to poor vocabulary and poor use of standard language. 
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Figure 3: Comparison of students’ skills at lower and higher taxonomic levels. 


c) Final state of the experimental and control group 


As mentioned already the main focus of this study was skill mastery at the end of the experiment, i.e. students’ 
progress in social, communication and working skills. Only after this had been established, could the question of 
whether project work enables greater development and progress in skill mastery than teaching focused on frontal 
instruction with explanations and text work be answered. 
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Table 8. _ Final state of skill mastery - lower level. 


GROUP Arithmetic Standard Test for difference of Test for homogeneity of 
mean deviation means variants 

SKILLS Xx S Boys Girls Boys Girls 
SOCIAL EG 4.686 0.411 
CG 4.055 0.588 
COMMUNICATION EG 4.024 0.853 

CG 2.944 0.596 3.476 0.001 3.704 0.063 
WORKING EG 4.566 0.356 
CG 4.037 0.494 


The test for difference of means (Table 8) showed that there was a significant difference between the control 
and the experimental group in students’ skill mastery (social, communication and working) at the lower level (Boys 
= 3.476; Girls = 0.001). The most progress was made in communication skills. 


Table9. —_ Final state of skill mastery - higher level. 


GROUP Arithmetic Standard Test for difference of Test for homogeneity of 
mean deviation means variants 

SKILLS x S Boys Girls Boys Girls 
SOCIAL EG 4.156 0.791 
CG 2.592 0.754 

COMMUNICATION EG 3.039 1.159 B2o1 231 0.000 gars 0.001 
CG 1.472 05a7 
WORKING EG 3.647 0.631 
CG 2.902 2.445 


The test for difference of means (Table 9) again showed a significant difference between the control and the ex- 
perimental group (B = 5.231; G = 0.000). Again, the most progress was made in communication and social skills. 


d) Progress in skill mastery with regard to gender 


Table 10. Progress in skill mastery with regard to gender. 


GROUP Arithmetic Standard Test for difference of Test for homogeneity of 
mean deviation means variants 
SKILLS X S Boys Girls Boys Girls 
SKILLS - INITIAL STATE Boys 2212 0.376 0.769 0.000 0.278 0.602 
Girls 2.982 0.559 
SKILLS - FINAL STATE Boys 3.171 0.669 0.991 0.000 0.278 0.602 
Girls 4.162 0.655 


It had been assumed that the most progress in skill development would be made by girls. The test for differ- 
ence of means (Table 10) showed that there was a significant difference in skill mastery progress with regard to 
student gender (Boys = 0,991; Girls = 0,000). This confirmed the assumption that progress is also dependent on 
gender. Girls achieved greater progress in skill mastery than boys. The difference between the initial and final state 
was 0.959 for boys and 1.18 for girls. 
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e) Progress in skill mastery with regard to final grade in S&T 

An assumption was also made that progress in skill mastery would be greater for those students whose final 
grade in S&T was higher. The test for difference of means (Table 11) showed that there was a significant difference 
in skill mastery progress with regard to final grade (Boys = 13.654; Girls = 0,000). This confirmed the expectation 
that progress in skill mastery is also dependent on the final grade. For students whose final grade was a C progress 
made was 0.922, for students with a B it was 0.955 and for those with an A it was 1.196. 


Table 11. Progress in skill mastery with regard to final grade. 
FINAL = 
Arithmetic mean Standard deviation Test for difference of means 
GRADE 

- S Boys Girls 

SKILL - INITIAL STATE C 2.045 0.192 
B 2.282 0.308 18.994 0.000 

A 3.005 0.563 

SKILL - FINAL STATE C 2.967 0.496 
2 3.937 0.605 13.654 0.000 

A 4.201 0.701 


An assumption was also made that progress in skill mastery would be greater for those students whose final 
grade in S&T was higher. The test for difference of means (Table 11) showed that there was a significant difference 
in skill mastery progress with regard to final grade (Boys = 13.654; Girls = 0,000). This confirmed the expectation 
that progress in skill mastery is also dependent on the final grade. For students whose final grade was a C progress 
made was 0.922, for students with a B it was 0.955 and for those with an A it was 1.196. 

The students’ combined progress in social, communication and working skills was also of interest. The level 
of students’ skill mastery is represented by an ordinate encompassing values from 1 to 5, where 1 means unsatis- 
factory, 1.5-2 satisfactory, 2.5-3 good, 3.5-4 quite good and 4.5-5 very good. As has already been established (at 
the lower and higher levels), the most progress was made in communication skills, and the progress was greater 
in the experimental group than in the control group (Figure 4). Thus, a conclusion can be made that the teaching 
approach influences skill development in students. 
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Figure 4; Progress in skill mastery with regard to group. 
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The effect size (d) was measured by deducting the average transfer grade in the control group from the average 
transfer grade in the experimental group. This was then divided by the combined standard deviation. According 
to Cohen, the effect size of +0.8 is large, +0.5 medium and +0.2 small, which is why planning principles with effect 
size 0.8 or more are of particular interest. 


Table 12. Effect size. 


Average Value 


Control group Experimental Combined standard Result 
(CG) group (EG) deviation 
Lower level 4.055 4.686 0.999 0.63 
SOCIAL SKILLS 

Higher level 2.892 4.156 1.546 0.81 
COMMUNICATION Lower level 2.944 4.024 1.450 0.74 
SKILLS Higher level 2.472 3.039 1.696 0.33 
Lower level 4.034 4.566 0.850 0.62 

WORKING SKILLS 
Higher level 2.902 3.647 3.076 0.24 

Average Value 3.216 4.019 
Discussion 


Planning lessons well is very important for the teacher since this is the only way to achieve the desired prog- 
ress in students (Cohen, 1988). This research mainly focused on developing and evaluating students’ skills in the 
frame of four generic competences. A large part of lesson planning is knowing all the generic competences and 
the specifics of each individual competence, especially which strategies, methods and types of lessons are being 
planned, since they fundamentally affect the development of a certain skill or competence. To sum up all the major 
findings presented in the research: 


1. Theresults show that, after the experiment, the mastery of skills at a lower and higher taxonomic level 
for a single generic competence is higher in students of the experimental group and mostly in learning 
and problem solving, interpretation, and verbal and written communication, where the appreciation 
is mostly due to successful methods of work. 

2. Boysand girls were, at a lower taxonomic level, the most proficient in social skills and the least proficient 
in communication skills. Boys achieved the most progress in mastering social and communication skills 
and girls in mastering working skills. 

3. From the calculated effect size (d) (Table 12), it can be concluded that the most progress in students’ 
skills was achieved in developing social and communication skills at a higher taxonomic level. A moder- 
ate effect was achieved in the field of working skills at a higher taxonomic level. 

4. |t was proven that in project work that is based on problem-based and research-based lessons (active 
students), better results were achieved in developing students’ skills than with frontal work (students 
are passive listeners). 


The relatively good result presented in Table 12 was surprising since the measured effect confirmed that greater 
development of students’ skills can be achieved by using PBL. The control group’s average value was 3.216, while 
that of the experimental group was 4.019. The most progress in skill mastery was achieved in developing social 
skills at a higher level, and in communication and working skills at a lower level, as had also been expected. 

That project based learning (PBL), which encompasses problem-based lessons, experimental and research 
work, and has a very strong effect on students’ process knowledge and indirectly on developing skills, is also proven 
by John Hattie’s results, who sieved through 900 results of meta-analysis, that included 52,600 studies and 240,000 
students (Hattie, 2013). He established that PBL have a strong effect (above 0.60) on the students’ achievement and 
also on cooperative learning, developing vocabulary, teaching learning strategies, solving problems and students’ 
prior achievements (prior knowledge). These are also the elements that we planned when implementing lessons 
and research. 
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Conclusions 


The results showed that project-based learning as a modern teaching method and part of the competence- 
based curriculum offers optimum development of students’ knowledge and skills. Progress depends on the gender 
and the preliminary score of the student. Not only is it important to develop certain skills i.e. competences, but 
one also needs to have suitable tools to test them. On the bases of Bloom’s, Simpson's and Dave's taxonomy for 
all three fields, cognitive, affective and psychomotor, the research developed a unified taxonomy (UTC — Unified 
Taxonomy of Competences) where acquired skills/competences in every field at a lower and higher taxonomic level 
are defined. For this purpose, an instrument for measuring students’ skills/competences was developed. 

The research was carefully planned, because it took place in a classroom with randomly chosen students in the 
presence of experts from the technical and pedagogical-didactical fields. A pilot study was carried out beforehand 
under the same conditions and where the students were of the same age. The developed instrumentation allowed 
evaluation of initial mastery of skills at both taxonomical levels, evaluation of final state and following the progress 
in mastering skills at both taxonomical levels. 

It is obvious that the results of the research will be of great help to teachers, as the instrumentation developed 
along with the measurement methodology will help improve the quality of examination and evaluation of students’ 
skills, not only in Science and Technology, but also in other areas of education, and not only in Slovenian schools, 
but also worldwide. The results and findings of this research could also have an impact on the consideration of 
amending the regulations on examination and assessment of knowledge in elementary school and integrating 
an assessment of student skills. 
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